Sharing user generated content for media searches

ABSTRACT

Disclosed are various embodiments for generating a search results interface for sharing user content for subsequent media searches of a physical object. In one example, a system comprises machine readable instructions stored in the a memory of a client device. The instructions cause the client device to at least transmit to a computing device a request for an image search of an object. The request comprises an image of the object. An image result is received from the computing device in response to the request for the image search. The instructions cause the client device to display an image search results interface. The image search results interface includes an image result from the image search and an upload user interface component. The upload user interface component is configured to display an input interface for uploading user content associated with the object in the image result.

BACKGROUND

Media searches provide another tool that enables users to find contentfor an object by using a media item that has the object as the searchinput. Accordingly, a user can perform a search for an object in a mediaitem without having to use text to describe the object. For example, auser can provide an image of a piece of furniture to search for othersimilar furniture items found in media items. However, these mediasearches provide search results of the media items with limited content.Accordingly, an improved system and method for sharing user generatedcontent for media searches is needed.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present disclosure can be better understood withreference to the following drawings. The components in the drawings arenot necessarily to scale, with emphasis instead being placed uponclearly illustrating the principles of the disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views.

FIGS. 1A-1C are pictorial diagrams of example user interfaces renderedby a client device according to various embodiments of the presentdisclosure.

FIG. 2 is a drawing of a networked environment according to variousembodiments of the present disclosure.

FIGS. 3A-3C are flowcharts illustrating examples of functionalityimplemented as portions of an application executed in a computingenvironment in the networked environment of FIG. 2 according to variousembodiments of the present disclosure.

FIG. 4 is a flowchart illustrating one example of functionalityimplemented as portions of an application executed in a computingenvironment in the networked environment of FIG. 2 according to variousembodiments of the present disclosure.

FIG. 5 is a schematic block diagram that provides one exampleillustration of a computing environment employed in the networkedenvironment of FIG. 2 according to various embodiments of the presentdisclosure.

DETAILED DESCRIPTION

Disclosed are various embodiments related to generating a search resultsinterface for sharing user content for subsequent media searches of aphysical object. The search results interface displays media files ordata streams identified based at least in part on a search for aphysical object contained within a media file.

Users may take a photograph or capture a video that includes a physicalobject. Users can then select the physical object to initiate a searchfor available media files or data streams (e.g., images, video, text,etc.) related to the selected physical object. A search resultsinterface can include one or media files that have been identified fromthe search of the object. The search results interface can include auser interface component (e.g., a button) that is configured to displayan input interface for receiving user content associated with the mediafiles in the search results interface. Subsequently, a different userthat performs a search of the same or similar physical object can accessthe user content uploaded by the previous user. As such, the variousindividuals can create and upload content that can be discovered byother individuals that execute a subsequent search for a media file thatincludes a particular object.

For example, if a user took a picture of a can of “Lemon-Lime Soda,”then any media files or data streams related to “Lemon-Lime Soda” wouldbe returned to the user. Likewise, a user could upload user content andlink or associate (e.g., “tag”) the user content with the media files ordata streams returned for the “Lemon-Lime Soda” object. For instance, auser could capture and upload an image of their can of “Lemon-LimeSoda”, thereby allowing the user's captured image or video to besurfaced to other users searching for media files related to “Lemon-LimeSoda.”

The search results interface can display a user interface component(e.g., an upload content button) for uploading the user content. Theuser interface component is displayed in association with the searchresults of the media files or data streams. Accordingly, upon reviewingthe media files in the search results interface, the user can select theuser interface component to upload user content to be stored inassociation with the media files or data streams for the object. In thefollowing discussion, a general description of the system and itscomponents is provided, followed by a discussion of the operation of thesame.

As one skilled in the art will appreciate in light of this disclosure,certain embodiments may be capable of achieving certain advantages. Forexample, the embodiments may be capable of an improved user interfacedesign by having an upload user interface component displayed on thesame interface (e.g., mobile application user interface, network page,etc.) as the search results from a visual search of an object. Theembodiments may be capable of the improved user interface reducing thenumber of user interfaces that a user has to navigate to in order toupload user content for the search results of a visual search.Additionally, the embodiments may be capable of an improved inputinterface that enables the user to share or upload a variety ofdifferent content for the visual search results of an object, and theimproved input interface may enable the adjustment of various settings(e.g., sharing methods, search triggers for an object) for the usercontent from a single interface.

With reference to FIG. 1A, shown is a pictorial diagram of an exampleuser interface rendered by a client device 100. FIG. 1A illustrates afirst example of a search results interface 103 on a display 106 of theclient device 100. The search results interface 103 includes a croppedmedia file 107 and a search results area 109. The cropped media file 107can be generated from a media file received or captured by the clientdevice 100. An object 113 in the media file is identified and the mediafile is cropped to focus on the identified object 113. The searchresults area 109 can include an upload user interface (UI) component 116and search results items 117. The upload UI component 116 is configuredfor uploading user content in association with the search results items117 identified from the search of the object 113. The upload UIcomponent 116 can be selected in order for the user to upload content,such as text, media items (e.g., an image file, a video file, an audiofile), hyperlinks, and other suitable material. After the content hasbeen uploaded, the uploaded content can appear in the search results ofsubsequent users executing a search based at least in part on the object113.

The search results items 117 can represent one or more media items 119that have been identified from the search to contain the object 113, asimilar object, and other material related to the object 113. In FIG.1A, a single media file has been identified from the search based atleast in part on the object 113. The search results interface 103 canalso include an edit object component 121. The edit object component 121is configured to allow the creator of the object 113 to edit varioussettings. For example, a user can edit the search triggers (e.g., imagesused to describe an object), text descriptors, and other suitablecomponents.

Next, with reference to FIG. 1B, shown is a pictorial diagram of anexample user interface rendered by the client device 100. FIG. 1Billustrates an input interface 130 that is displayed upon a selection ofthe upload UI component 116 (FIG. 1A) from the search results interface103. In some embodiments, upon selecting the upload UI component 116, auser interface prompt can be displayed to name the entry for the usercontent. In FIG. 1B, the entry name has been entered as “New Entry.” Themedia item 119 from the search result is displayed near the top. Theinput interface 130 also includes a share component 139 for sharing thecontent entry with other user accounts established on an applicationplatform. Additionally, the input interface 130 includes an invitecomponent 142 for sharing the content entry with other users that do nothave a user account on the application platform.

The input interface 130 also includes a search trigger area 133 and acontent entry area 136. The search trigger area 133 includes a searchtrigger media item 145 and a manage component 148. The search triggermedia item 145 represents one or more media files that have the object113. The media files can be used as a basis for generating variousdescriptors for the object 113, and these descriptors can be usedexecute the search for the object 113. The manage component 148 can beconfigured to add or remove media items 119 that include the object 113.A user may select the manage component 148 in order to add differentviews of the object 113. For example, if the object 113 is a car in themedia item 119, the user may select the manage component 148 in order toadd different views of the exterior and/or the interior of the car. Thedifferent views can be used for providing search results of the carobject 113.

The content entry area 136 includes a text window 151 and an attachmentcomponent 154. The text window 151 can be used for entering structuredor unstructured text, a hyperlink, and other suitable text. In someembodiments, the hyperlink provided may be from a hyperlink to a mediaitem (e.g., an image or a video) posted on a website, such as a socialmedia platform. The attachment component 154 can be configured to uploadan image file, a video file, an audio file, and other suitable mediacontent.

After the user has uploaded the content, the content can be stored inassociation with the object 113 for subsequent users to have access intheir search results for the object 113. As such, the upload UIcomponent 116 enables the transition from the search results interface103 to the input interface 130, which provides for a simpler userexperience for uploading content related to the search results of anobject 113 than having to navigate through various user interfaces or avariety of different applications, such as when a user may have toswitch from one application to another or one user interface to another.

Moving on the FIG. 1C, shown is a pictorial diagram of an example userinterface rendered by the client device 100. FIG. 1C illustrates anexample of a second search results interface 160 on the client device100. In FIG. 1C, the user content from FIG. 1B has been uploaded and isavailable to be accessed by a subsequent user. Accordingly, the secondsearch results interface 160 includes user content that was uploadedfrom the input interface 130 by a previous user.

As illustrated in FIG. 1C, the second search results interface 160includes search results 109 with various search results items 117. Thesearch results items 117 which can include multiple media items 119a-119 d and one or more user reviews 120. In one example, the new mediaitems 119 b-119 d and the user review 120 may have been uploaded by theinput interface 130. Additionally, a portion of the user review 120 isshown.

In some embodiments, the media items 119 can be configured in differentarrangements. For example, the media items 119 displayed in FIG. 1C arearranged in a media item collage. In other examples, the media items 119can be arranged in a continuous feed, in which the media items are in asingle vertical column, in which the user can scroll up or down to viewadditional media items. Also, in FIG. 1C, the new media items 119 b-119d can represent a cropped image of a larger media item displayed at awebsite (e.g., a product page, a social media post, etc.).

In some examples, the search results items 117 can be arranged, sorted,or filtered according to various criteria. For example, the criteria caninclude filtering or prioritizing based at least in part on most likes(e.g., received from a social media platform), popularity, relevancy,most views, most recent, closest location of the search results items117 to the client device 100, and other suitable factors. Third partymetadata associated with the search results items 117 can be received inorder to filter, sort, or arrange the search results items 117 based atleast in part on one or more factors. For example, website metadata fromretailers, social media platforms, and other websites can providemetadata associated with the media items 119.

For example, the media item 119 b and media item 119 c may representimages from social media posts. The media item 119 b may have received alarger quantity of likes from a social media platform than the mediaitem 119 c. As a result, the media item 119 b is displayed higher thanthe media item 119 c.

With reference to FIG. 2 , shown is a networked environment 200according to various embodiments. The networked environment 200 includesa computing environment 203 and a client device 100, which are in datacommunication with each other via a network 206. The network 206includes wide area networks (WANs) and local area networks (LANs). Thesenetworks can include wired or wireless components or a combinationthereof. Wired networks can include Ethernet networks, cable networks,fiber optic networks, and telephone networks such as dial-up, digitalsubscriber line (DSL), and integrated services digital network (ISDN)networks. Wireless networks can include cellular networks, satellitenetworks, Institute of Electrical and Electronic Engineers (IEEE) 802.11wireless networks (i.e., WI-FI®), BLUETOOTH® networks, microwavetransmission networks, as well as other networks relying on radiobroadcasts. The network 206 can also include a combination of two ormore networks 206. Examples of networks 206 can include the Internet,intranets, extranets, virtual private networks (VPNs), and similarnetworks.

The computing environment 203 may include, for example, a servercomputer or any other system providing computing capability.Alternatively, the computing environment 203 may employ a plurality ofcomputing devices that may be arranged, for example, in one or moreserver banks or computer banks or other arrangements. Such computingdevices may be located in a single installation or may be distributedamong many different geographical locations. For example, the computingenvironment 203 may include a plurality of computing devices thattogether may include a hosted computing resource, a grid computingresource, or any other distributed computing arrangement. In some cases,the computing environment 203 may correspond to an elastic computingresource where the allotted capacity of processing, network, storage, orother computing-related resources may vary over time.

Various applications or other functionality may be executed in thecomputing environment 203 according to various embodiments. Thecomponents executed on the computing environment 203, for example,include an object search application 209, and other applications,services, processes, systems, engines, or functionality not discussed indetail herein.

The object search application 209 is executed to identify objects 113(FIG. 1A) in image or video files and search for files related to theidentified objects 113. Accordingly, the object search application 209may rely on various computer vision and object identification techniquesto identify unique objects 113 in an image or video file. The objectsearch application 209 may then search for related files using one ormore of the various techniques further described herein. The objectsearch application 209 may also be configured, in some embodiments, toallow users to upload files to the data store 213, link the file to oneor more objects 113, and set various permissions identifying which otherusers are permitted to view, consume, or otherwise access the uploadedfile.

Various data is stored in a data store 213 that is accessible to thecomputing environment 203. The data store 213 may be representative of aplurality of data stores 213, which can include relational databases,object-oriented databases, hierarchical databases, hash tables orsimilar key-value data stores, as well as other data storageapplications or data structures. The data stored in the data store 213is associated with the operation of the various applications orfunctional entities described below. This data can include objectrecords 216, media file records 219, user accounts 223, and potentiallyother data.

The object record 216 represents information that is related to ordescribes an object 113. Accordingly, the object record 216 can includean object identifier 226, object data 229, one or more point clouds 233,and user content 235. Other data may also be stored in the object record216 as required by various embodiments of the present disclosure.

The object identifier 226 corresponds to a primary key that allows theobject record 216 to be uniquely identified in the data store 213 by theobject search application 209. An example of an object identifier 226would include a numeric identifier generated when the object record 216is initially created. If sequential numbers were used, this would allowfor each object record 216 to have a unique object identifier 226.

The object data 229 represents information about the object 113. Objectdata 229 can include, for example, the name of the object 113, the ownerof the object 113 (e.g., property owner), notes or facts about theobject 113, the price of the object 113 (e.g., consumer goods), ratingsor reviews of the object 113 (e.g., ratings of goods by other users),and any other data appropriate for the particular object 113corresponding to the object record 216. Object data 229 may be added tothe object record 216 when the object record 216 is first created, or itmay be updated by one or more users at a later point in time. In someembodiments, limits may be placed on which user accounts 223 are allowedto modify the object data 229.

The point clouds 233 represent collections of points that define theshape or contour of an object 113. Multiple point clouds 233 may bestored within an object record 216 to represent different views of thesame object 113 (e.g., views from different angles or perspectives). Theobject search application 209 can use a point cloud 233 stored in anobject record 216 to determine whether an object 113 in an image orvideo is the same as an object 113 corresponding to the object record216.

The user content 235 represents content that is uploaded by the user tobe associated with an object 113 in a media file 236 from the visualsearch results. For instance, upon a display of search results from thevisual search, the user of the client device 100 can add their owncontent to one of the media items 119 in the search results. Subsequentusers can view the user content 235 upon performing a visual search forthe object 113. The user content 235 can include user generated text,media files 236 (e.g., image file, video file, audio file, etc.),hyperlinks, and other suitable user-generated content.

The media file record 219 represents information that is related to ordescribes a media file 236 related to an object record 216 for aparticular object 113. The media file 236 can represent either adiscrete file, a data stream, or a combination thereof. Accordingly,references to the term media file 236 can include references to a file,a data stream, or a combination thereof as may be appropriate for aparticular implementation. The media file record 219 can include anobject identifier 226 for the corresponding object record 216 of eachobject 113 identified as present in the media file 236. Accordingly, ifmultiple objects 113 are identified as present in the media file 236,then multiple object identifiers 226 may be stored as part of the mediafile record 219. The media file record 219 may also include one or morepermissions 239, one or more locations 241 linked to or associated withthe media file 236, and file data 243.

The permissions 239 of the media file record 219 define which useraccounts 223 are permitted to access or interact with the media file 236and the manner in which access or interaction is permitted. For example,the permissions 239 may specify which user accounts 223 are allowed toview or consume the media file 236, which user accounts 223 are allowedto modify the media file 236, which user accounts 223 are allowed toremove or delete the medial file 236 and/or the corresponding media filerecord 219. Other permissions 239 may also be specified as appropriatefor particular embodiments of the present disclosure. In some instances,the permissions 239 may be applied to individual user accounts 223 orgroups of user accounts 223 (e.g., a “Public” or “Everybody” groupencompassing all users, a “family” group comprising user accounts 223belonging to a group of users that are members of the same family,etc.).

The location 241 can represent geographic information about the mediafile record 219. For example, the location 241 can be used to storeinformation about where the media file 236 was created, a location 241to be associated with a media file 236 (e.g., associating the city ofWashington, D.C. with a media file 236 depicting the WashingtonMonument), or similar information. One or more locations 241 can also bestored in the media file record 219. For example, a media file record219 could include the actual coordinates for the Washington Monument,the geohash or geocode representing the National Mall, and a geohash orgeocode representing the city of Washington, D.C. itself.

File data 243 represents metadata about the media file 236 itself. Suchmetadata can include the name of the media file 236, the type of file(e.g., image, video, etc.), the format of the media file 236 (e.g.,MPEG-4 video encoding, JPEG image, etc.), the size of the media file236, the date the media file 236 was created, the date the media file236 was last modified, or the date that the media file 236 was uploaded.Other information may also be stored within the file data 243 as may beappropriate for particular embodiments of the present disclosure.

User accounts 223 represent individual users of the object searchapplication 209. Each user account 223 may generally correspond to oneuser (e.g., an individual or an organization), although a user may havemultiple user accounts 223 (e.g., a personal and a professionalaccount). Accordingly, a user account 223 may include a user identifier246, information about the group membership 249 of the user account 223,and potentially other information.

The user identifier 246 represents a unique identifier that allows for auser account 223 to be distinguished from other user accounts 223. Insome implementations, this may correspond to a username selected by auser when the user account 223 is first created. In otherimplementations, the user identifier 246 may be automatically generated(e.g., a sequential number). In some implementations, multiple useridentifiers 246 may be used for efficiency purposes. For example, someimplementations may rely on a username to represent the user account 223to other users while using an automatically assigned number whenchecking for applicable permissions 239, which increases the speed ofthe check by relying on simpler integer operations of the computingdevice.

The group membership 249 lists all of the groups of user accounts 223for which the specific user account 223 is a member. A user account 223may belong to one, none, or multiple groups, each of which may begranted various permissions.

The client device 100 is representative of a plurality of client devicesthat may be coupled to the network 206. The client device 100 mayinclude, for example, a processor-based system such as a computersystem. Such a computer system may be embodied in the form of a personalcomputer (e.g., a desktop computer, a laptop computer, or a similardevice), a mobile computing device (e.g., personal digital assistants,cellular telephones, smartphones, web pads, tablet computer systems,music players, portable game consoles, electronic book readers, andsimilar devices), media playback devices (e.g., media streaming devices,BluRay® players, digital video disc (DVD) players, set-top boxes, andsimilar devices), a videogame console, or other devices with likecapability. The client device 100 may include one or more displays 106,such as liquid crystal displays (LCDs), gas plasma-based flat paneldisplays, organic light emitting diode (OLED) displays, electrophoreticink (“E-ink”) displays, projectors, or other types of display devices.In some instances, the display 106 may be a component of the clientdevice 100 or may be connected to the client device 100 through a wiredor wireless connection. Additionally, the client device 100 can includea camera 251 for capturing images and/or video of an object 113 for avisual search.

The client device 100 may be configured to execute various applicationssuch as a client application 253 or other applications. The clientapplication 253 may be executed in a client device 100, for example, tosend one or more media files 236 to the object search application 209 toinitiate a search for related media files 236 or to add the media file236 to a media file record 219 in the data store 213. To this end, theclient application 253 may include, for example, a browser, a dedicatedapplication, or other executable software that can cause a userinterface 103/130/160 to be rendered on the display 106. The userinterface 103/130/160 may include a network page, an application screen,or other user mechanism for obtaining user input or presenting output tothe user. The client device 100 may be configured to executeapplications beyond the client application 253 such as, for example,email applications, social networking applications, word processors,spreadsheets, or other applications.

Next, a general description of the operation of the various componentsof the networked environment 200 is provided. It should be noted thatthis general description is meant to serve to illustrate the principalsof the present disclosure. One or more features described in thisgeneral description may or may not be present in any particularembodiment of the present disclosure.

To begin, a user may capture an image or video of an object 113 and savethe captured image or video as a media file 236. In some instances, thelocation 241 where the media file 236 was created may also be recorded.For example, the user may use the client application 253 to cause his orher mobile device to capture an image of a bottle of juice. The clientapplication 253 may then prompt the user to upload the media file 236and save it to the data store 213. Assuming the user choses to uploadthe media file 236, the client application 253 then sends the media fileto the object search application 209.

The object search application 209 then uses one or more computer visiontechniques to recognize individual objects 113 within the media file 236and generate a corresponding point cloud 233 for each identified object113. The object search application 209 then searches the object records216 within the data store 213 to determine whether any object records216 include a point cloud 233 that matches one or more of the pointclouds 233 generated from the media file 236. The object searchapplication 209 can then send object data 229 and the object identifier226 for each matching object record 216 to the client application 253.

After matching media files 236 have been identified, the object searchapplication 209 can transmit the matching media files 236 to the clientdevice 100. The client application can display the search resultsinterface 103, which can include the matching media files 236 in thesearch results area 109. The search results interface 103 can alsoinclude the upload UI component for uploading user content.

For example, assuming the user takes a picture of a bottle of “FarmFresh Juice” and performs a search based at least in part on the “FarmFresh Juice” bottle as the object 113, the search results interface 103can display one or more media items 119 that include the “Farm FreshJuice” bottle.

At this point, the user can select the upload UI component 116 to uploaduser content 235 for the search results items 117 associated with the“Farm Fresh Juice” bottle. The client application 253 can display aninput interface 130 for receiving the user content 235. For example, theuser may enter text for a user review, such as “Love this new favor bythe juice company.” The user may also upload a media file 236 (e.g., animage or video file) by selecting the attachment component 154. Theuploaded user content 235 may be a picture or a video taken by the userof their bottle. The client application 253 can transmit or upload themedia file 236 and the text to the object search application 209. Afterthe user content 235 (e.g., the media item 119 and the text) has beenuploaded, the object search application 209 can store the user content235 in association with the “Farm Fresh Juice” bottle object 113. Assuch, the user content 235 is stored in association with the object 113,such as with the object record 216 and/or the media file record 219 ofthe “Farm Fresh Juice” bottle.

Subsequently, a different user can perform a second search based atleast in part on the media file 236 that includes the “Farm Fresh Juice”bottle. The object search application 209 can transmit search resultsitems 117 to the client application 253 of a second client device 100.The client application 253 can display a second search results interface160 that includes the search results items 117 and the user content 235uploaded by the previous user of the first client device 100.

Referring next to FIG. 3A, shown is a flowchart that provides oneexample of the operation of a portion of the object search application209 according to various embodiments. It is understood that theflowchart of FIG. 3A provides merely an example of the many differenttypes of functional arrangements that may be employed to implement theoperation of the portion of the object search application 209 asdescribed herein. As an alternative, the flowchart of FIG. 3A may beviewed as depicting an example of elements of a method implemented inthe computing environment 203 (FIG. 2 ) according to one or moreembodiments.

Beginning with box 303, the object search application 209 receives afirst media file 236 (FIG. 2 ) from a client device 100 (FIG. 2 ). Forexample, a client application 253 executing on the client device 100 maymake use of one or more functions made available by the object searchapplication 209 through an application programming interface (API) toupload the first media file 236 to the object search application 209.

Proceeding to box 306, the object search application 209 uses one ormore computer vision techniques to identify individual objects 113within the first media file 236. For example, the object searchapplication 209 may use various feature detection approaches such asedge detection, corner detection, blob detection, ridge detection,affine invariant feature detection, or similar approaches to identify acollection of features that define an object. This collection offeatures could include a point cloud 233, a collection of edges, acollection of corners, or similar grouping of features that can be usedto uniquely identify the object 113 from other objects 113 in the firstmedia file 236.

Moving on to box 309, the object search application 209 determineswhether the collection of features of an object 113 in the first mediafile 236 matches a collection of features of a known object 113. Forexample, if the object search application 209 identifies a point cloud233 of a first object 113, the object search application 209 maydetermine whether the point cloud 233 matches a second point cloud 233stored in an object record 216 corresponding to a known object 113. Ifno match is found, execution ends. However, if a match is found,execution can instead proceed to box 313.

Referring next to box 313, the object search application 209 may searchfor one or more media files 236 related to the identified object 113.For example, the object search application 209 can determine whether theobject identifier 226 of the object record 216 matching the identifiedobject 113 matches the object identifiers 226 stored in any media filerecords 219. Proceeding next to box 316, the object search application209 determines whether a second media file 236 related to the object 113identified in the first media file 236 has been found. If a match isfound, then execution can proceed to box 319. Otherwise, execution mayend.

Moving on to box 319, the object search application 209 may then checkone or more permissions 239 of the media file record 219 to determinewhether the client device 100 or a user account 223 (FIG. 2 ) associatedwith the client device 100 is allowed to access (e.g., view, edit,share, etc.) the matching media file 236. For example, the object searchapplication 209 may determine whether the client device 100 or the useraccount 223 associated with the client device 100 is explicitly listedas having permission to access the media file 236. Likewise, the objectsearch application 209 may also check whether the client device 100 orthe user account 223 is included in a group of client devices 100 or agroup of user accounts 223 that is listed as having permission to accessthe media file 236. If the client device 100 or the user account 223associated with the client device 100 or the user account 223 associatedwith the client device 100 has permission to access the media file 236,then execution proceeds to box 323. Otherwise, execution of the processmay end.

Referring next to box 323, the object search application 209 sends thesecond media file 236 to the client application 253 executing on theclient device 100. In some embodiments, execution of this process mayend. However, in other embodiments, one or more of the previouslydescribed steps may be repeated. For example, portions of the previouslydescribed process may be repeated in order for the object searchapplication 209 to search for multiple media files 236 that may berelated to the object 113 identified in the first media file 236.

Referring next to FIG. 3B, shown is a flowchart that provides oneexample of the operation of a portion of the object search application209 according to various embodiments. The flowchart of FIG. 3B providesan example of functionality related to displaying a search resultsinterface 103/160 (FIGS. 1A and 1C). It is understood that the flowchartof FIG. 3B provides merely an example of the many different types offunctional arrangements that may be employed to implement the operationof the portion of the object search application 209 as described herein.As an alternative, the flowchart of FIG. 3B may be viewed as depictingan example of elements of a method implemented in the computingenvironment 203 (FIG. 2 ) according to one or more embodiments.

Beginning with box 350, the object search application 209 can receive arequest to execute a visual search for an object 113 identified in amedia file 236. The process for executing the visual search is describedin FIG. 3A.

Proceeding to box 351, the object search application 209 can transmitsearch results data and search results interface data to the clientdevice 100. The search results data can include the search result items117. The search results interface data can include data for generatingthe search results interface 103, such as the upload UI component 116,the edit object component 121, and other suitable components.

Moving on to box 353, the object search application 209 can receive anindication that the user has manipulated (e.g., clicked, tapped, orselected) the upload UI component 116 for uploading user content 235. Insome embodiments, the object search application 209 can receive datafrom the client device 100 that indicates an entry name for the usercontent 235, for example, as “New Entry” is displayed in FIG. 1B. Theentry name data can be retrieved from a user interface displayed on theclient device 100.

Next, with regard to box 356, the object search application 209 cantransmit input interface data to the client device 100. The inputinterface data can include the search trigger area 133, the managecomponent 148, the content entry area 136, the media attachmentcomponent 154, and other suitable user interface features.

Referring next to box 359, the object search application 209 can receivethe user content 235 from the client device 100. The user content 235can include text, a hyperlink, a media file 236 (e.g., an image file, avideo file, an audio file), and other suitable user content. In someembodiments, the hyperlink can be activated to display a website (e.g.,product webpage or a social media posting) that has a media item 119.

Moving on to box 362, the object search application 209 can store theuser content 235 for the object 113. In some embodiments, the usercontent 235 can be stored in association with the object record 216and/or the media file record 219 for the object. Then, the depictedprocess ends, although the object search application 209 could continueexecution to perform other functions.

Referring next to FIG. 3C, shown is a flowchart that provides oneexample of the operation of a portion of the client application 253according to various embodiments. The flowchart of FIG. 3C can representfunctionality for search results interface that displays a searchresults from an image search and displays one or more user interfacecomponents for uploading user content associated with an object of thesearch results. It is understood that the flowchart of FIG. 3C providesmerely an example of the many different types of functional arrangementsthat may be employed to implement the operation of the portion of theclient application 253 as described herein. As an alternative, theflowchart of FIG. 3C may be viewed as depicting an example of elementsof a method implemented in the computing environment 203 (FIG. 2 )according to one or more embodiments.

Beginning with box 375, the client application 253 can use a camera 251to capture a media file 236 (e.g., an image or a video) of an object 113for an image search. In some examples, the media file 236 captured canbe manipulated into a cropped media file 107 of the object 113. In somescenarios, this step is omitted and the client device 100 retrieves amedia file 236 from its memory.

Next, with regard to box 378, the client application 253 can transmit tothe computing environment 203 (e.g., the object search application 209)a request for an image search (e.g., a visual search) of the object 113.The request can include the media file 236 that displays the object 113.

Referring to box 381, the client application 253 can receive from thecomputing environment 203 one or more image results (e.g., search resultitems 117) in response to the request for the request for the imagesearch of the object 113. The search result items 117 can include one ormore media items 119, text (e.g., user reviews 120), and other relatedmedia items from the image search.

Moving on to box 384, the client application 253 can display a searchresults interface 103. The search results interface 103 can display theimage results from the image search, in which the object 113 was asearch input parameter (e.g., a search trigger). The search resultsinterface 103 can also include an upload UI component 116 for uploadinguser content 235 for the object 113. The search results items 117 caninclude one or more media files 236 that contain the object 113 or asimilar object 113.

Next, with regard to box 387, the client application 253 can receive anindication of a selection of the upload UI component 116 by the user. Insome embodiments, a user interface can be displayed for naming a newentry of user content 235. For example, the user may name the new entry“New Entry,” as shown in FIG. 1B.

At box 390, the client application 253 can display the input interface130 for receiving the user content 235. The name of the new entry can bedisplayed on the input interface 130. The input interface 130 can alsodisplay a share component 139, an invite component 142, a search triggerarea 133, and a content entry area 136. The input interface 130 canreceive user content 235, such as text, media items, and other suitablecontent from the user.

Referring to box 393, the client application 253 can transmit the usercontent 235 to the computing environment 203. The user content 235 canbe stored in association with the object 113 for subsequent imagesearches of the object 113 by other client devices 100.

In some embodiments, the search results interface 103 can be a mediafeed for a user account 223 of a social media platform. The media feedcan display a plurality of media items 119 in the search results area109. For example, a user may request an image search of a company logodisplayed in the media file 236. Upon receiving the image results, amedia feed can be displayed with search results items 117 of variousproducts associated with the company logo. The search results items 117may be media files 236 that have been uploaded by different clientdevices 100 of different users. The media files 236 may be imagescaptured by the different client devices of one or more products thathave the company logo. Then, the depicted process ends, although theclient application 253 could continue execution to perform otherfunctions

Referring next to FIG. 4 , shown is a flowchart that provides oneexample of the operation of a portion of the object search application209 according to various embodiments. The flowchart of FIG. 4 canrepresent functionality for a user to access user content from a searchresults interface, in which the user content was uploaded to thecomputing environment 203 by another user. It is understood that theflowchart of FIG. 4 provides merely an example of the many differenttypes of functional arrangements that may be employed to implement theoperation of the portion of the object search application 209 asdescribed herein. As an alternative, the flowchart of FIG. 4 may beviewed as depicting an example of elements of a method implemented inthe computing environment 203 (FIG. 2 ) according to one or moreembodiments.

Beginning with box 403, it is assumed the object search application 209has received user content 235 for one or more users. For example, theuser content 235 may be uploaded and associated to one or more objects113 according to the flowchart of FIG. 3B. At this stage, the objectsearch application 209 may receive a first media file 236 (FIG. 2 ) froma second client device 100 (FIG. 2 ). In this example, the second clientdevice 100 is different from a first client device 100 that has uploadedthe user content 235 stored in association with the object 113.

Next, with regard to box 406, the object search application 209 can useone or more computer vision techniques to identify individual objects113 within the first media file 236. For example, the object searchapplication 209 may use various feature detection approaches such asedge detection, corner detection, blob detection, ridge detection,affine invariant feature detection, or similar approaches to identify acollection of features that define an object. Moving on to box 409, theobject search application 209 can identify the collection of features ofan object 113 in the first media file 236 that matches a collection offeatures of a known object 113. In some embodiments, the functionalitycan be similar to the functionality described in FIG. 3A.

Referring next to box 413, the object search application 209 can searchfor one or more media files 236 related to the identified object 113.For example, the object search application 209 can determine whether theobject identifier 226 of the object record 216 matching the identifiedobject 113 matches the object identifiers 226 stored in any media filerecords 219, in which the functionality can be similar to thefunctionality described in FIG. 3A.

Next, with regard to box 416, the object search application 209 cansearch for user content 235 associated with the object 113.Additionally, in some embodiments, the user content 235 can be retrievedbased at least in part on an association with the media files 236 thathave the object 113. As such, the user content 235 can be retrievedbased at least in part on a search for the media file record 219, theobject identifier 226, the object record 216, and other suitable means.In some examples, the object search application 209 can search the usercontent 235 for content that is similar or matches an object 113 of asearch.

Moving on to box 419, the object search application 209 can send thesecond media file 236 and the user content 235 to the client device 100.In some examples, the user content 235 can be retrieved for a feedinterface of a social media platform. The feed interface can include acontinuous feed of user content 235 associated with a particular object113. For example, a user can request a visual search of a company logo.The company logo can be identified as the object 113. The object searchapplication 209 can transmit for a display in the continuous feed usercontent 235 that has been uploaded to multiple client devices 100 onbehalf of various users. Then, the depicted process ends, although theobject search application 209 could continue execution to perform otherfunctions.

With reference to FIG. 5 , shown is a schematic block diagram of thecomputing environment 203 according to an embodiment of the presentdisclosure. The computing environment 203 includes one or more computingdevices 503. Each computing device 503 includes at least one processorcircuit, for example, having a processor 506 and a memory 509, both ofwhich are coupled to a local interface 512. To this end, each computingdevice 503 may comprise, for example, at least one server computer orlike device. The local interface 512 may comprise, for example, a databus with an accompanying address/control bus or other bus structure ascan be appreciated.

Stored in the memory 509 are both data and several components that areexecutable by the processor 506. In particular, stored in the memory 509and executable by the processor 506 is the object search application209, and potentially other applications. Also stored in the memory 509may be a data store 213 and other data. In addition, an operating systemmay be stored in the memory 509 and executable by the processor 506.

It is understood that there may be other applications that are stored inthe memory 509 and are executable by the processor 506 as can beappreciated. Where any component discussed herein is implemented in theform of software, any one of a number of programming languages may beemployed such as, for example, C, C++, C#, Objective C, Java®,JavaScript®, Perl, PHP, Visual Basic®, Python®, Ruby, Flash®, or otherprogramming languages.

A number of software components are stored in the memory 509 and areexecutable by the processor 506. In this respect, the term “executable”means a program file that is in a form that can ultimately be run by theprocessor 506. Examples of executable programs may be, for example, acompiled program that can be translated into machine code in a formatthat can be loaded into a random access portion of the memory 509 andrun by the processor 506, source code that may be expressed in properformat such as object code that is capable of being loaded into a randomaccess portion of the memory 509 and executed by the processor 506, orsource code that may be interpreted by another executable program togenerate instructions in a random access portion of the memory 509 to beexecuted by the processor 506, etc. An executable program may be storedin any portion or component of the memory 509 including, for example,random access memory (RAM), read-only memory (ROM), hard drive,solid-state drive, USB flash drive, memory card, optical disc such ascompact disc (CD) or digital versatile disc (DVD), floppy disk, magnetictape, or other memory components.

The memory 509 is defined herein as including both volatile andnonvolatile memory and data storage components. Volatile components arethose that do not retain data values upon loss of power. Nonvolatilecomponents are those that retain data upon a loss of power. Thus, thememory 509 may comprise, for example, random access memory (RAM),read-only memory (ROM), hard disk drives, solid-state drives, USB flashdrives, memory cards accessed via a memory card reader, floppy disksaccessed via an associated floppy disk drive, optical discs accessed viaan optical disc drive, magnetic tapes accessed via an appropriate tapedrive, and/or other memory components, or a combination of any two ormore of these memory components. In addition, the RAM may comprise, forexample, static random access memory (SRAM), dynamic random accessmemory (DRAM), or magnetic random access memory (MRAM) and other suchdevices. The ROM may comprise, for example, a programmable read-onlymemory (PROM), an erasable programmable read-only memory (EPROM), anelectrically erasable programmable read-only memory (EEPROM), or otherlike memory device.

Also, the processor 506 may represent multiple processors 506 and/ormultiple processor cores and the memory 509 may represent multiplememories 509 that operate in parallel processing circuits, respectively.In such a case, the local interface 512 may be an appropriate networkthat facilitates communication between any two of the multipleprocessors 506, between any processor 506 and any of the memories 509,or between any two of the memories 509, etc. The local interface 512 maycomprise additional systems designed to coordinate this communication,including, for example, performing load balancing. The processor 506 maybe of electrical or of some other available construction.

Although object search application 209, and other various systemsdescribed herein may be embodied in software or code executed by generalpurpose hardware as discussed above, as an alternative the same may alsobe embodied in dedicated hardware or a combination of software/generalpurpose hardware and dedicated hardware. If embodied in dedicatedhardware, each can be implemented as a circuit or state machine thatemploys any one of or a combination of a number of technologies. Thesetechnologies may include, but are not limited to, discrete logiccircuits having logic gates for implementing various logic functionsupon an application of one or more data signals, application specificintegrated circuits (ASICs) having appropriate logic gates,field-programmable gate arrays (FPGAs), or other components, etc. Suchtechnologies are generally well known by those skilled in the art and,consequently, are not described in detail herein.

The flowcharts of FIGS. 3A, 3B, 3C, and 4 show the functionality andoperation of an implementation of portions of the object searchapplication 209. If embodied in software, each block may represent amodule, segment, or portion of code that comprises program instructionsto implement the specified logical function(s). The program instructionsmay be embodied in the form of source code that comprises human-readablestatements written in a programming language or machine code thatcomprises numerical instructions recognizable by a suitable executionsystem such as a processor 506 in a computer system or other system. Themachine code may be converted from the source code, etc. If embodied inhardware, each block may represent a circuit or a number ofinterconnected circuits to implement the specified logical function(s).

Although the flowcharts of FIGS. 3A, 3B, 3C, and 4 show a specific orderof execution, it is understood that the order of execution may differfrom that which is depicted. For example, the order of execution of twoor more blocks may be scrambled relative to the order shown. Also, twoor more blocks shown in succession in FIGS. 3A, 3B, 3C, and 4 may beexecuted concurrently or with partial concurrence. Further, in someembodiments, one or more of the blocks shown in FIGS. 3A, 3B, 3C, and 4may be skipped or omitted. In addition, any number of counters, statevariables, warning semaphores, or messages might be added to the logicalflow described herein, for purposes of enhanced utility, accounting,performance measurement, or providing troubleshooting aids, etc. It isunderstood that all such variations are within the scope of the presentdisclosure.

Also, any logic or application described herein, including the objectsearch application 209, that comprises software or code can be embodiedin any non-transitory computer-readable medium for use by or inconnection with an instruction execution system such as, for example, aprocessor 506 in a computer system or other system. In this sense, thelogic may comprise, for example, statements including instructions anddeclarations that can be fetched from the computer-readable medium andexecuted by the instruction execution system. In the context of thepresent disclosure, a “computer-readable medium” can be any medium thatcan contain, store, or maintain the logic or application describedherein for use by or in connection with the instruction executionsystem.

The computer-readable medium can comprise any one of many physical mediasuch as, for example, magnetic, optical, or semiconductor media. Morespecific examples of a suitable computer-readable medium would include,but are not limited to, magnetic tapes, magnetic floppy diskettes,magnetic hard drives, memory cards, solid-state drives, USB flashdrives, or optical discs. Also, the computer-readable medium may be arandom access memory (RAM) including, for example, static random accessmemory (SRAM) and dynamic random access memory (DRAM), or magneticrandom access memory (MRAM). In addition, the computer-readable mediummay be a read-only memory (ROM), a programmable read-only memory (PROM),an erasable programmable read-only memory (EPROM), an electricallyerasable programmable read-only memory (EEPROM), or other type of memorydevice.

Further, any logic or application described herein, including objectsearch application 209, may be implemented and structured in a varietyof ways. For example, one or more applications described may beimplemented as modules or components of a single application. Further,one or more applications described herein may be executed in shared orseparate computing devices or a combination thereof. For example, aplurality of the applications described herein may execute in the samecomputing device 503, or in multiple computing devices in the samecomputing environment 203. Additionally, it is understood that termssuch as “application,” “service,” “system,” “engine,” “module,” and soon may be interchangeable and are not intended to be limiting.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each be present.

It should be emphasized that the above-described embodiments of thepresent disclosure are merely possible examples of implementations setforth for a clear understanding of the principles of the disclosure.Many variations and modifications may be made to the above-describedembodiment(s) without departing substantially from the spirit andprinciples of the disclosure. All such modifications and variations areintended to be included herein within the scope of this disclosure andprotected by the following claims.

Therefore, the following is claimed:
 1. A non-transitorycomputer-readable medium storing machine readable instructions that,when executed by a processor of a computing device, cause the computingdevice to at least: capture an image of an object using a camera;transmit to a remote computing device a request for a visual search ofthe object, the request comprising the image of the object; receive fromthe remote computing device an image result in response to the requestfor the visual search, wherein the image result comprises the object;display a visual search results interface that includes the image resultfrom the visual search and an upload user interface component; anddisplay an input interface in response to a selection of the upload userinterface component, the input interface having a management componentthat is configured for adding a media item to a set of items stored inassociation with the object and configured for removing a particularmedia item from the set of items stored in association with the object,the set of items being accessible for a subsequent visual search by aplurality of client devices.
 2. The non-transitory computer-readablemedium of claim 1, wherein the media item comprises at least one of, animage file, or an audio file.
 3. The non-transitory computer-readablemedium of claim 1, wherein the visual search results interface comprisesa cropped media file of the image and a plurality of search resultitems.
 4. The non-transitory computer-readable medium of claim 1,wherein the input interface is configured to receive a hyperlink of themedia item, wherein upon activation of the hyperlink the media item isdisplayed, the hyperlink is transmitted to the remote computing in orderfor the hyperlink to be stored in association with the object.
 5. Thenon-transitory computer-readable medium of claim 1, wherein the inputinterface includes an invite component and a sharing component, thesharing component being configured to send an entry of the media item toa different user account, and the invite component being configured toinvite a non-registered user to create a user account and view the entryof the media item.
 6. A system, comprising: a client device comprising aprocessor and a memory; and machine readable instructions stored in thememory that, when executed by the processor, cause the client device toat least: transmit to a computing device a request for an image searchof an object, the request comprising an image of the object; receivefrom the computing device an image result in response to the request forthe image search, wherein the image result comprises the object; displayan image search results interface that includes the image result fromthe image search and a selectable component; and display an inputinterface for uploading user content associated with the object in theimage result in response to receiving a selection of the selectablecomponent, the input interface having a management component that isconfigured for adding a media item to a set of items stored inassociation with the object and configured for removing a particularmedia item from the set of items stored in association with the object.7. The system of claim 6, wherein the input interface comprises a textwindow for receiving a text input.
 8. The system of claim 6, wherein themachine readable instructions, when executed by the processor, cause theclient device to at least: transmit the user content to the computingdevice, wherein the user content is stored in association with theobject in the image result, wherein the user content is configured to bedisplayed in association with a second image result for a second clientdevice.
 9. The system of claim 6, wherein the input interface comprisesan attachment component that is configured for uploading the media itemin association with the object in the image result.
 10. The system ofclaim 7, wherein the input interface includes the image result from theimage search results interface and a sharing component, the sharingcomponent being configured to send an entry of the user content to adifferent user account.
 11. The system of claim 10, wherein the imageresult is a cropped image of a larger image that includes the object.12. The system of claim 6, wherein the user content comprises at leastone of text, an image file, an audio file, or a hyperlink of a webpage.13. The system of claim 6, wherein the image search results interfaceincludes a search results area that includes the image result from theimage search.
 14. The system of claim 6, wherein the image searchresults interface is a media feed for a user account on a social mediaplatform.
 15. A computer-implemented method, comprising: identifying, bya client device, an image of an object; transmitting, by the clientdevice, a request for an image search of the object to a computingdevice, the request comprising the image of the object; receiving, bythe client device, an image result from the computing device in responseto the request, the image result comprising the object; displaying, bythe client device, an image search results interface that includes theimage result and a user interface component; and displaying, by theclient device, an input interface for uploading user content associatedwith the object in the image result in response to receiving a selectionof the user interface component, the input interface having a managementcomponent that is configured for adding a media item to a set of itemsstored in association with the object and configured for removing aparticular media item from the set of items stored in association withthe object.
 16. The method of claim 15, wherein the input interfacedisplays a search trigger media item, the search trigger media itemrepresenting a basis for generating at least one search descriptor forthe object, associated with the object.
 17. The method of claim 16,further comprising: transmitting, by the client device, the user contentto the computing device, wherein the user content is stored inassociation with the object in the image result, and the user content isconfigured to be displayed for a subsequent image second for a secondclient device.
 18. The method of claim 15, wherein identifying the imageof the object comprises: capturing, by the computing device, the imageof the object using a camera.
 19. The method of claim 15, wherein theuser content comprises at least one of text, an image file, an audiofile, or a hyperlink of a webpage.
 20. The method of claim 15, whereinthe image search results interface is a media feed for a user account ona social media platform, wherein the media feed includes a plurality ofmedia items retrieved from a plurality of different social mediaplatforms.